Still-image photography and full-motion video have been continually improved over the years. From Nièpce's photo etchings beginning in 1822, through the development by Nièpce and Daguerre of photography using silver compounds, through the development of inexpensive and widely available cameras, through instant photography continuing with modern digital photography, the ability to record still and moving images has become more and more widespread until it is now nearly universal. Many people document every aspect of their lives by gathering and distributing images of interesting or noteworthy events.
These days, photography is often casual, carried out under widely varying lighting conditions, by photographers who are not interested in making meticulous camera adjustments. Nonetheless, such photographers would like their photographs and videos to look attractive and to resemble the subject, particularly in terms of such properties as white balance. White balance is a camera setting that adjusts for lighting in order to make white objects appear white in a photo or video. This is more difficult than it might seem to be, due to the fact that the light cast from two different sources may differ substantially in terms of spectral content and color. Ambient light is very rarely truly white in nature. The spectral content of a light source may be referred to as color temperature. The light from an incandescent or halogen bulb, for example, may emphasize the orange and yellow regions of the spectrum, while a fluorescent light may emphasize the blue region of the spectrum.
A proper white balance setting in a camera will prevent a white bed sheet in a photo from appearing to be orange, for example, while the sheet is being illuminated by candlelight. Because it is particularly important that neutral colors (such as white or gray) appear correctly, white balance may also be referred to as color balance or gray balance. White balance is a measure of the relative intensities of each color component of an image (such as the red, green, and blue additive primary colors of light). Color constancy is a characteristic of the human visual system—the human brain provides a relatively constant color perception over a significant range of wavelength variation of light illuminating the subject. Artificial mechanisms for capturing images, on the other hand, need to incorporate mechanisms for adjustment and correction in order to achieve such constancy.
Automatic white balance (AWB) and auto exposure (AE) algorithms employed in the camera imaging pipeline are critical to the color appearance of digital pictures and videos. The goal of AWB algorithms is to provide the color constancy feature of the human visual system, wherein the perceived color of an object remains substantially constant under different conditions of illumination. Thus, AWB algorithms have to determine, from the image content itself, the necessary color correction to perform in response to the current illumination. For video, the AWB algorithm is usually employed on a per-frame basis to account for scene changes. Auto exposure algorithms aim to adjust the camera's exposure time to minimize overexposed and underexposed areas. This functionality is needed since cameras are only capable of measuring a limited range of the total illumination in the scene (i.e., cameras have limited dynamic range). To capture bright areas, a shorter exposure time is needed; otherwise, the image will be overexposed or saturated (white). On the other hand, in dark areas a longer exposure time is needed; otherwise, the image will be underexposed or black. The goal of AE algorithms is to find the optimal exposure for the scene being captured. When capturing video, this is typically done using a control loop that analyzes incoming video frames and provides estimates of the best exposure time to capture subsequent frames.
AWB algorithms attempt to estimate the illumination in an image by making some assumptions regarding the distribution of colors in the image. Then, these AWB algorithms correct the image as if it was taken under standard illumination. Some of the most well known and widely used AWB algorithms are the white patch algorithm, which assumes that the brightest point in the image is white (maximum reflectance); and the gray world algorithm which assumes that the average reflectance of a scene is achromatic. Other AWB algorithms are based upon certain assumptions regarding the content of an image.
AE algorithms typically analyze exposure statistics of an image (e.g., an intensity histogram) to determine how much the exposure time should be changed in order to obtain “optimal” statistics for the next captured image. Different criteria for defining the “optimal” exposure may be used, and those typically follow a heuristic approach. Illustrative heuristic approaches include having the mean intensity value close to a neutral gray, prioritizing shadow areas, constraining the optimization to a region pre-set by the user, or prioritizing skin tones.
Conventional AWB and AE algorithms do not provide optimal performance in many real-world scenarios. These AWB and AE algorithms are executed on a per-frame basis while capturing video sequences. This often leads to undesirable tonal fluctuations and exposure variations in video frames, as is commonly observed in the context of mobile cameras.